Skip to content

feat(i18n): automated translation pipeline with machine review gate#623

Draft
Timur Tukaev (tym83) wants to merge 35 commits into
mainfrom
feat/i18n-pipeline
Draft

feat(i18n): automated translation pipeline with machine review gate#623
Timur Tukaev (tym83) wants to merge 35 commits into
mainfrom
feat/i18n-pipeline

Conversation

@tym83

Copy link
Copy Markdown
Contributor

Summary

Builds the automated translation pipeline on top of the i18n proof-of-concept in #593. Where #593 established the multi-language site structure and a few hand-checked pages, this turns translation into a repeatable, reviewable process: it discovers what changed in the English source, translates it through a machine review gate, and opens a weekly PR for a maintainer to merge. Nothing reaches production without that merge.

Stacked on #593 — please review and merge that first; this PR targets poc/i18n-multilang and will retarget to main once #593 lands.

What

  • Change detection. source_digest (sha256 of the English source, the same convention hack/check-i18n.sh already enforces) drives a worklist of missing/stale pages per language. Scope is the latest docs version only, plus recent blog posts — older docs versions are noindex, so translating them would spend budget on pages search engines ignore.
  • Per-page review gate. Each page runs translate → back-translate (meaning-drift check) → two virtual native reviewers (a technical editor for fluency, a Cozystack maintainer for technical correctness) → a bounded revise loop. Code, shortcodes, and inline code are masked during translation and restored after, so they stay byte-for-byte. caption/alt text inside shortcodes is translated (it renders to readers); src/width are not.
  • Honest stamping. A page that clears the gate is stamped translation_review: auto-reviewed; one that runs out of revise rounds with findings still open is stamped auto-reviewed-with-findings and its findings are posted to the weekly PR so a maintainer can triage them. Only a human sets ratified.
  • Reader-facing disclosure. Documentation pages carry a machine-translation banner linking to the English original until a native speaker marks them ratified. The banner is docs-only by design — the blog, marketing pages, and homepage hero do not carry it.
  • Cadence. The runner translates daily (to use each day's model budget) but accumulates into one i18n/week-<ISO week> branch, so maintainers review and merge a single translation PR per week. publish_mode: pr_only — CODEOWNERS and branch protection are untouched.
  • Sample output. Two real pipeline translations of the same docs page (ru + de, docs/v1.5/getting-started/install-kubernetes.md) are included so quality can be judged directly.

Why

The English docs are the source of truth and change constantly; native localization takes months. This lets translated docs ship and be indexed immediately (publish-then-ratify), with the banner keeping that honest, while native review happens asynchronously and is tracked per page. The review gate is not a substitute for native ratification — both virtual reviewers are the same model as the translator, so the gate measures self-consistency and catches the obvious failures, no more. It is deliberately conservative: fail-closed on unparseable reviewer verdicts, refuses to write a page with a dropped code block, and never publishes without a maintainer merge.

Notes for reviewers

  • Auth. Bootstrapped on a maintainer's Claude subscription via the Claude Agent SDK (no metered billing). The intent is to move the backlog burst to an organization-owned API key; that is a one-line config switch (auth: api-key), no code change. See README "Ownership and continuity".
  • Backlog size. 183 pages × 6 languages ≈ 1084 jobs. Measured on the pilot, a page is ~3 min when it clears the gate and ~11 min when it runs the full revise loop, so the first pass is best done on an API key rather than a personal subscription.
  • es/pt-br are translated by the pipeline but not yet declared in hugo.yaml (no content shipped). Declaring a language before its content exists publishes empty indexable pages; a test enforces the ordering.
  • Tests for the pure functions: python3 hack/i18n/test_i18n.py.

Preview

The included sample pages render under /ru/docs/v1.5/getting-started/install-kubernetes/ and /de/… in the deploy preview, each with the machine-translation banner.

@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 292e7c62-9f78-47ed-a8d6-e0888f78bb19

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/i18n-pipeline

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an automated, machine-reviewed translation pipeline for the Cozystack website, adding localization scripts, prompts, style guides, and tests under hack/i18n/, alongside initial support for Spanish and Brazilian Portuguese. It also integrates a machine-translation disclaimer banner into the documentation layout and adds hreflang alternates for SEO. Feedback on these changes highlights a non-standard Hugo API usage (.Sites.Default) in the translation banner, a duplicate entry in .gitignore, and formatting conflicts in the translation and revision prompts where the instructions ask for key: value lines instead of the JSON format expected by the parsing script.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +31 to +32
{{- $src := .Sites.Default.Language.Lang -}}
{{- if and (ne .Lang $src) (ne .Params.translation_review "ratified") -}}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

.Sites.Default is not a standard Hugo API and may cause template evaluation errors depending on the Hugo version. The standard and safe way to retrieve the default content language is .Site.DefaultContentLanguage.

Suggested change
{{- $src := .Sites.Default.Language.Lang -}}
{{- if and (ne .Lang $src) (ne .Params.translation_review "ratified") -}}
{{- $src := .Site.DefaultContentLanguage -}}
{{- if and (ne .Lang $src) (ne .Params.translation_review "ratified") -}}

Comment thread .gitignore Outdated
Comment on lines +31 to +36
# i18n pipeline venv (bootstrapped by hack/i18n/run-daily.sh)
.venv-i18n/

# i18n pipeline: per-run artifacts, not content
hack/i18n/last-run-findings.md
.venv-i18n/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The directory .venv-i18n/ is duplicated in the .gitignore file (on lines 32 and 36). Removing the duplicate entry keeps the file clean.

# i18n pipeline venv (bootstrapped by hack/i18n/run-daily.sh)
.venv-i18n/

# i18n pipeline: per-run artifacts, not content
hack/i18n/last-run-findings.md

Comment on lines +43 to +46
===FRONTMATTER===
<the translated values, one `key: value` per line, same keys, same order>
===BODY===
<the translated body>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The system prompt instructs the model to output the front matter as key: value lines, but the translation pipeline (translate.py via _FM_PROTOCOL) expects a JSON object. This conflict can confuse the model and lead to protocol parsing errors. Updating the prompt to match the JSON protocol will improve reliability.

Suggested change
===FRONTMATTER===
<the translated values, one `key: value` per line, same keys, same order>
===BODY===
<the translated body>
===FRONTMATTER===
<the translated values as a JSON object, same keys, same order>
===BODY===
<the translated body>

Comment on lines +21 to +24
===FRONTMATTER===
<corrected key: value lines, same keys, same order>
===BODY===
<corrected body>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the translation prompt, the revision system prompt instructs the model to output key: value lines for the front matter, which conflicts with the JSON object format expected by _FM_PROTOCOL in translate.py. Updating this to instruct the model to return a JSON object will prevent protocol errors during the revision loop.

Suggested change
===FRONTMATTER===
<corrected key: value lines, same keys, same order>
===BODY===
<corrected body>
===FRONTMATTER===
<corrected JSON object, same keys, same order>
===BODY===
<corrected body>

Timur Tukaev (tym83) and others added 20 commits July 17, 2026 23:17
In-tree localization engine for cozystack.io (hack/i18n/), building on the
existing source_digest freshness convention (hack/check-i18n.sh):

- worklist.py: diff detector (missing/stale pages via source_digest)
- translate.py: Claude Opus translator with glossary, per-language style
  guides, protected code/shortcodes/URLs, SEO front-matter transcreation
- ahrefs_keywords.py: per-locale SEO keyword maps (optional, degrades
  gracefully without AHREFS_API_KEY)
- i18n-translate.yml: nightly + dispatch; PR then auto-merge (publish-then-review)
- config.yaml: languages, model routing (all Opus), scope, blog cutoff
  (last ~2 months), publish mode

Add Spanish (es) and Portuguese-BR (pt-br): hugo.yaml + production mounts,
i18n/es.toml + i18n/pt-br.toml (key parity verified).

hreflang alternates + x-default in head-end.html. CODEOWNERS exempts
content/<lang>/ so the pipeline auto-merges translations while code stays gated.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
… OAuth, daily runner

Rework the pipeline per maintainer decisions:
- Auth: OAuth subscription (Max), not a metered API key — bare Anthropic()
  client resolves the logged-in credential; ANTHROPIC_API_KEY warns.
- Machine review gate on EVERY page before publish: translate -> back-translate
  + meaning-drift compare -> two native reviewers (technical editor for
  fluency, Cozystack maintainer for technical correctness) -> revise-if-needed,
  bounded by review.max_rounds. Pages written atomically only after passing.
- Daily-until-limit: run stops cleanly on 429 and resumes next day; adds
  hack/i18n/run-daily.sh (commit + daily PR + auto-merge).
- New prompts: back-translate(-compare), review-editor, review-maintainer,
  revise. config.yaml gains auth/rate_limit/back_translation/review sections
  and a translation_review front-matter stamp.
- Workflow switched off metered API key to optional CLAUDE_CODE_OAUTH_TOKEN;
  local daily runner is the primary path.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
The base anthropic SDK cannot use a Claude subscription (metered API only), so
the model-call layer now uses the Claude Agent SDK (claude-agent-sdk), which
reads CLAUDE_CODE_OAUTH_TOKEN from `claude setup-token`.

- translate.py: call() runs a single-turn, tool-less Agent SDK query
  (allowed_tools=[], max_turns=1) via asyncio; hard-fails if ANTHROPIC_API_KEY
  is set (it would shadow the subscription); warns if CLAUDE_CODE_OAUTH_TOKEN
  is missing; rate-limit detection maps to a clean daily stop.
- run-daily.sh: require CLAUDE_CODE_OAUTH_TOKEN, forbid ANTHROPIC_API_KEY,
  install claude-agent-sdk.
- workflow: CLAUDE_CODE_OAUTH_TOKEN secret + install claude-agent-sdk and the
  claude-code CLI.
- config.yaml/README: document the Agent SDK subscription path + setup.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
Verified on a real machine: the Agent SDK authenticates off an existing
`claude` CLI login — CLAUDE_CODE_OAUTH_TOKEN is only needed headless/CI. So:

- run-daily.sh no longer hard-fails without the token; it requires only that
  ANTHROPIC_API_KEY is unset (the money guard) and that some subscription
  credential exists (claude login or token).
- run-daily.sh bootstraps a venv (.venv-i18n) and runs the pipeline from it —
  distro Pythons are PEP 668 externally-managed, so plain pip install fails.
- translate.py: drop the misleading missing-token warning.
- gitignore the venv.

Smoke-tested end to end: claude-agent-sdk 0.2.121, single-turn tool-less query
against claude-opus-4-8 returns text over the subscription.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
Ahrefs API v3 access is an expensive, separately-licensed add-on, and the
English source is already SEO-optimized during authoring — the translation
inherits that intent. The translate prompt still transcreates title/description
to read naturally per locale, so nothing is lost by removing the keyword step.

Removes ahrefs_keywords.py, keyword-maps/, the ahrefs config block, the
keyword-hint injection in translate.py, the workflow step, and the
AHREFS_API_KEY secret. No secrets are required for translation now beyond the
subscription credential.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
…injection

Adversarial review found the pipeline did not actually work and that its
central claim was not enforced by code. Fixes:

P0 publish: run-daily.sh used "git diff --quiet", which ignores untracked
files — every new translation is untracked, so it reported "nothing to publish"
forever while source_digest on disk made the page look fresh the next day
(silent permanent no-op). Now uses "git status --porcelain" scoped to
content/<lang>, branches from a fresh origin/main (days no longer stack), and
honours publish_mode (pr_only was documented but never implemented).

P0 gate honesty: _parse_verdict failed OPEN — refusals, prose and truncated
JSON all silently passed, making the gate unfalsifiable; and the revise loop
stamped pages that never cleared it as auto-reviewed. Now fails CLOSED
(unparseable = revise), a missing prompt file hard-fails instead of sending a
reviewer out with no instructions, and pages are stamped honestly
(auto-reviewed vs auto-reviewed-with-findings) with per-run counters.

P0/P1 correctness: reject replies without the ===BODY=== protocol instead of
writing the model's preamble as page content; verify every protected
placeholder survived (a dropped fence silently deleted a code block);
hard-fail on unparseable front matter instead of dropping slug/date/aliases.

P1 scope: the docs version is now read from hugo.yaml (latest_version_id)
instead of hardcoded v1.4 while latest is v1.5 — 164 of 352 pages were being
translated into a noindex'd version. Scope drops 352 -> 188 pages
(2096 -> 1128 jobs). fnmatch replaced with pathlib semantics so the config no
longer lies about what it matches.

P1 security: the workflow interpolated github.event.inputs into the shell,
letting anyone with write access exfiltrate the subscription token; inputs now
go through env and are quoted.

P1 auth: auth is now a config choice (oauth-subscription | api-key) so the
project can move from a maintainer's subscription to an org-owned key without
code changes.

P2: read the whole front matter for source_digest (a long one made pages look
permanently stale and re-translate daily); unknown --lang errors instead of
silently doing nothing; stable YAML dump; drop dead config/code; stale v1.2
default in head-end.html.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
Resolves the governance objection from review: nothing about CODEOWNERS or
branch protection changes any more, and no machine output reaches the
production site without a maintainer merging it.

- CODEOWNERS: reverted to the original single rule. The previous exemption for
  content/<lang>/ would have removed required review from those paths for ANY
  author, not just the bot — on a tree where goldmark renders unsafe HTML.
- publish_mode: pr_only is now the default and is actually honoured.
- Cadence: the runner still runs DAILY (each day's quota is used in full), but
  the week's output accumulates onto one i18n/week-<ISO week> branch, so
  maintainers review and merge a single translation PR per week instead of a
  stream. A new week branches fresh from origin/main.
- Docs: README documents the daily-run/weekly-PR split, the honest
  auto-reviewed vs auto-reviewed-with-findings stamps, and an "Ownership and
  continuity" section stating the intent to move from a maintainer's
  subscription to an organization-owned API key.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
Second review pass over the pipeline. The findings that mattered were all
silent-failure modes: the run would look successful while producing wrong or
no output.

Nested front matter. The landing page renders its hero and cards from
taglines[], benefits[], and features[]; only top-level keys were translated,
so regenerating a localized homepage would copy the English hero back over a
hand-translated one and drop locale-only keys (seo:, l10n:) that natives had
added. Front-matter values are now addressed by path, translated at any depth,
and applied onto a deep copy, with target-only keys re-attached. The
front-matter wire format moves from "key: value" lines to JSON, which also
fixes multi-line descriptions being shredded by the line parser.

Fail closed. A "revise" verdict with an empty findings list no longer passes
the gate; latest_docs_version raises instead of returning None (which made the
scope filter a no-op and pulled in every old docs version); a duplicated
placeholder is rejected like a dropped one; a missing front-matter key is
rejected rather than shipping a page whose body is translated but whose hero
is still English.

run-daily.sh no longer stashes in the working tree — it requires a clean
dedicated clone and restores the starting branch on exit. Stashing wrote
conflict markers into translations and could pop someone else's stash. Also
fixes a BrokenPipeError that killed the run before the first page, and stops
treating normal check-i18n.sh staleness as a publish blocker.

Tests cover the pure functions. One of them corrected a docstring that claimed
path globbing narrowed `*.md` to the repo root; it does not, and the config
does not want it to.

The GitHub workflow is removed: the pipeline runs from a maintainer's clone,
and a workflow implied CI-hosted credentials we deliberately do not use.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
Localized pages are indexed from day one so readers get the docs now instead of
after a native review that may take months. That trade is only honest if the
page says what it is, and until now nothing did.

Add a translation-banner partial that carries a machine-translation notice and
links to the English original. The rule is fail-safe: it shows on any
non-English page unless the front matter says `translation_review: ratified`.
Keying off the field's presence instead would have exempted every page from the
i18n PoC, which is machine output with no such field — precisely the pages the
notice exists for. Wired into the same four layouts as version-banner, so docs,
blog, and pages are covered; the homepage is deliberately left out, since
covering it means overriding the theme's home layout and a warning across a
marketing hero costs more than it buys. Strings are localized in all seven
locales.

Stop declaring es and pt-br in hugo.yaml. Declaring a language with no content
does not build nothing: Hugo emits /es/, /es/tags/, /es/categories/, /es/topics/,
/es/article_types/ and /es/404.html regardless, all `index, follow` and
self-canonical — twelve empty but indexable pages across the two, which is the
thin content the rest of this work is careful to avoid. They are commented out
in hugo.yaml and config.yaml together, with the enable order documented:
translate first, then declare, in the PR that carries the content.

The pipeline now writes `l10n: mt`, reusing the site's existing convention for
how a page was localized rather than inventing another field. README documents
what each of the three front-matter fields means and who reads it.

README also now says plainly what the review gate is not: both "native
reviewers" are the same model as the translator with different prompts, which
measures self-consistency, not native ratification. `auto-reviewed` must not be
read as "a human checked this" — only a human sets `ratified`, and only that
drops the banner. Adds a rollback section, since a pipeline that publishes to
production needs a documented way to stop.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
… scope

The preferred-terms list covered de, ru, es, and pt-br. es and pt-br are not
enabled; zh-cn and hi are, and had zero entries — the two live languages with
nothing keeping their terminology consistent had the coverage the disabled ones
got. Adds all eight terms for both.

`Tenant` and `tenant` are in do_not_translate and preferred respectively, which
reads as a contradiction and is not one: the capitalized term is the API kind a
reader has to match against `kind: Tenant` in a manifest, the lower-case one is
an ordinary noun that has to be translated or the prose is unreadable. Both the
glossary and the translate prompt now say so, and a test asserts any other
overlap between the lists is a real contradiction.

Excludes oss-health/** — five pages that are front matter plus
`layout: oss-health-app`, a dashboard rendered client-side from live English
data. Translating them wraps localized chrome around an English dashboard.
They also carry the site's only `lede` field, a user-visible string that was
not in the translatable-key list and would have shipped in English on every
localized copy; dropping these pages is what makes that list true rather than
approximately true.

Tests now run against the real content tree: that config.yaml and hugo.yaml
agree on which languages are enabled, and that no unrecognized user-visible
front-matter key has appeared. The second is the one that would have caught
`lede`.

Scope is now 183 pages x 4 languages = 720 jobs.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
run-daily.sh forwards the same "$@" to both scripts, so `--limit 3` made
worklist.py exit with a usage error. The preview was wrapped in
`|| true`, so it failed silently — and only when --limit was passed, which is
exactly the pilot-run case it exists to preview.

An unknown --lang printed "all languages up to date" and exited 0, since the
filter simply matched nothing. It is now an error.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
A pilot run produced a page stamped `auto-reviewed-with-findings` and no
record anywhere of what the findings were. The stamp told a maintainer that
something was wrong and gave them nothing to act on, which makes the
distinction between it and `auto-reviewed` close to useless.

translate_page now returns the findings still open on the final round. The
runner prints them per page and writes a markdown report, which run-daily.sh
posts as a PR comment.

A comment, not the PR body: the report file holds only the last run, while the
PR accumulates a week of daily runs — rewriting the body each day would drop
the earlier days' findings. Comments accumulate on their own, so the thread
becomes the week's log.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
Two pilot runs on the same release blog post: de cleared the gate in 2m54s, ru
took 11m24s through the revise loop. Per-page cost is dominated by whether the
revise loop runs, not by the page — which puts the 720-job backlog at roughly
35-140 hours of wall clock before daily limits are even considered, and is the
concrete reason the backlog needs an organization API key rather than a
personal subscription.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
Commenting es and pt-br out of the pipeline config stopped them being
translated at all, which is not the goal — the goal is that they are not
*served* until they have content. The two lists answer different questions and
were wrongly kept identical.

config.yaml (what gets translated) now covers all six languages. hugo.yaml
(what gets built and served) still declares four; es and pt-br are declared in
the PR that lands their content.

The invariant is one-directional, and the test now says so: translated but not
declared is how a language starts; declared but not translated means the site
serves something nothing keeps fresh. A second test asserts every declared
language actually has content, which is the failure that started this — Hugo
emits ~6 indexable pages for a declared language with an empty content tree.

Backlog is now 183 pages x 6 languages = 1084 jobs.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
A virtual reviewer on the ru pilot caught this, which is the gate doing exactly
its job: protect() masked each {{< figure >}} shortcode wholesale, so its
caption (rendered as visible text under the image) and alt (screen readers,
SEO) stayed English on every localized page — four English captions in a row on
a translated release post.

protect() now splits a shortcode into protected structure and exposed
visible-text attribute values (caption, alt, title). src, width, delimiters,
and param names stay byte-for-byte; the values translate like ordinary prose.
Shortcodes with no such attribute are still masked wholesale, unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
Product decision: the machine-translation notice belongs where a wrong
technical detail is costly — an operator running a translated command — not on
the blog, marketing pages, or the homepage hero. Wired into docs/baseof.html
only; removed from page/single, blog/baseof, and resources/list. The partial's
guard is unchanged and layout-agnostic, so coverage is now purely a question of
which layouts call it.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
A pilot on the v1.4 release blog post (63 opaque placeholders, 49 of them inline
code) had the model non-deterministically drop five of them. The gate correctly
refused to write the page — a silently deleted code block is worse than a
retry — but a page that fails this way on every daily run would never publish
while re-spending quota each day.

translate.py now retries a page a few times within the run on a protocol error,
since the loss is non-deterministic and usually clears on a fresh attempt. A
page that fails every attempt is skipped as before and stays in the worklist;
README documents that a reproducible failure means translate that page by hand.

Adds --path to translate one exact page (pilot runs, or re-translating a single
page after editing its source).

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
Two pipeline outputs for reviewers to judge quality on, both of the same source
page (docs/v1.5/getting-started/install-kubernetes.md) so they can be compared
side by side. Both cleared the review gate (translation_review: auto-reviewed);
inline code, the {{% ref %}} shortcode, and front-matter structure are preserved
verbatim, and each carries the machine-translation banner shown on docs pages.

These are real, reviewable output — not fixtures. The English source is
untouched.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <timur.tukaev@aenix.io>

Signed-off-by: tym83 <6355522@gmail.com>
…rklist

_docs_out_of_scope excluded every docs page not under docs/<latest>/, which
also dropped docs/_index.md — the version-picker landing that #593 translates
per language and records a source_digest for. The freshness lint would then
flag drift on that page with no pipeline path to refresh it.

Narrow only the versioned subtrees (docs/<ver>/...) to the latest version;
pages directly under docs/ are version-agnostic and stay in scope. Adjust the
leak-in guard test accordingly and cover the landing explicitly.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
The docs version-picker landing strings were added to the active-language
i18n files on poc/i18n-multilang (#593). es and pt-br live only on this
pipeline branch, so the rebase onto that work left them without those keys —
the parity lint flags them as missing. Add the Spanish and Portuguese
translations so all seven language files stay in parity.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
Base automatically changed from poc/i18n-multilang to main July 20, 2026 09:06

@kvaps Andrei Kvapil (kvaps) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed this PR's own diff (36 files) against its base, with #593 now merged. Ran hack/i18n/test_i18n.py (49 pass) and hack/check-i18n.sh (clean). The design is careful and the README is unusually honest about what the gate is and isn't — nearly all of my comments are about the runner, not the translation logic.

Blocking: the daily runner wedges on its second run

run-daily.sh translates on whatever branch the clone happens to be on, publishes on a different one, and the EXIT trap restores the original branch (hack/i18n/run-daily.sh:79-81, 115-126). In order:

  1. Day 1 commits translations onto i18n/week-<N> and pushes. The trap then checks out START_REF, which deletes those files from the working tree — they now only exist on the week branch.
  2. Day 2 therefore sees them as missing again (build_worklist checks the filesystem, lib.py:197-208) and re-translates the exact same pages, spending a day of quota redoing day 1.
  3. Publishing then fails hard: the re-translated files are untracked, origin/i18n/week-<N> already contains them, and git checkout -B "$BRANCH" origin/$BRANCH (run-daily.sh:121) aborts with "The following untracked working tree files would be overwritten by checkout". Under set -e the run dies leaving the tree dirty, so every later run stops at the clean-tree preflight (run-daily.sh:74-78).

I reproduced this in a scratch repo following the exact command sequence: day 2 exits 1 at the checkout and leaves ?? content/ru/ behind. The first run of a new week hits the same wall through the origin/main path once the previous week's PR is merged.

Root cause is that the working tree is never synced to the remote — git fetch (line 115) updates refs only, so both the English sources and the already-published translations stay frozen at clone time, and worklist correctness depends on that tree.

Suggestion: check out the week branch before translating — fetch, git checkout -B "$BRANCH" origin/$BRANCH (or origin/main), then run translate.py, then commit in place. That makes the tree the branch the work belongs to, keeps day 2's worklist accurate, removes the untracked-file collision, and refreshes content/en in the same move.

Worth fixing

The revise loop's last round is never re-reviewed. translate.py:271-319 — on the final iteration a page with findings is revised and the loop then exits. The text written is post-revision, but findings (stamped, returned, and posted to the weekly PR) describe the pre-revision text. So the finding list in the PR comment doesn't correspond to the pages actually shipped, and the last Opus call for each failing page produces output nothing ever checks. Either re-check after the last revise, or skip revising on it.

"A page is only written once it passes the gate" (README.md:35, config.yaml:115) contradicts the code and the rest of the README: pages that exhaust the revise loop are written, stamped -with-findings (translate.py:342-346, README.md:249-252). The diagram at README.md:13-33 has no edge for that path either. This is the sentence a reviewer leans on, so it's worth making it say what happens.

auto_merge exists. The default is pr_only and the claim in the description holds today, but run-daily.sh:170-173 will call gh pr merge --auto --squash after a one-word config change, with no second signal required. If "nothing reaches production without a maintainer merge" is meant as a property rather than a default, consider dropping that branch — nothing in this PR uses it.

Failures in the publish stage are swallowed. gh pr create ... || true and gh pr comment ... || true (lines 152-154, 167). A failed gh call leaves commits pushed, no PR opened, and exit 0 — indistinguishable from a good run in cron output.

Unpinned dependencies. pip install --quiet claude-agent-sdk pyyaml runs on every invocation (line 34), upgrading in place on a machine that holds a maintainer's CLAUDE_CODE_OAUTH_TOKEN and gh credentials. A pinned requirements.txt costs nothing here.

An i18n problem already visible in the shipped strings

The banner builds one sentence out of four keys plus link text (layouts/partials/translation-banner.html:39-42). That hardcodes English word order and punctuation, and two of the six languages already read wrong:

  • hi: "देखें अंग्रेज़ी मूल" — Hindi is verb-final; the natural order is "अंग्रेज़ी मूल देखें".
  • de / ru: "Einen Fehler entdeckt? eröffnen Sie ein Issue." and "Нашли ошибку? создайте issue." — lowercase after a question mark.

One key per sentence, with the link as a placeholder, lets a translator move it. Worth noting these UI strings never pass through the pipeline's review gate — only content/ does.

Smaller things

  • .gitignore:32 and :36.venv-i18n/ listed twice.
  • README.md:93 says 183 × 4 = 720 jobs; config.yaml lists six languages and the description says ~1084.
  • Nothing removes a translation whose English source was deleted, and check-i18n.sh only checks digests of files that exist, so orphans accumulate silently.
  • blog_since: "2026-05-17" is a fixed date that will quietly rot into "nothing new is in scope" — the config comment already suggests computing now-60d.

On the security posture and the gate

No workflow is added, so the usual automation risks don't apply: no pull_request_target, no repo-held secret, no untrusted checkout. Model calls use allowed_tools=[] and max_turns=1 (translate.py:117-122), so the translator has no filesystem or tool access, and git add -- content/<lang> genuinely bounds what a run can commit. What's left is that everything runs from one person's machine on their personal subscription, with their gh token and their DCO sign-off on machine-generated commits — and that the human merge gate is procedural, since no maintainer can meaningfully review Hindi or Chinese docs in a bulk weekly PR. The README says as much and the banner is the right mitigation; I'm noting it, not objecting.

The gate is real machinery rather than a stamp: verdict parsing fails closed (translate.py:149-172), an explicit "revise with no findings" verdict still blocks (lines 288-290, 300-302), and a lost or duplicated placeholder is a hard refusal to write (lines 213-219). Its ceiling is the one the description already admits — same model, so it measures self-consistency. Worth being precise that it never blocks publication, only stamps: it's a triage classifier, not a gate.

Requesting changes for the runner bug alone; the rest is small. Noting for the record that this is a draft and was stacked on #593, which has since merged.

Timur Tukaev (tym83) and others added 5 commits July 20, 2026 17:41
The runner translated on whatever branch the clone was on and only switched
to the week branch to publish, while the EXIT trap restored the starting
branch afterwards. Day 1's translations then vanished from the working tree
(they existed only on the week branch), so day 2 saw the same pages as
missing, re-translated them on a day's quota, and crashed on checkout: the
re-translated files were untracked and already present on origin's week
branch. Under set -e that left a dirty tree, and every later run stopped at
the clean-tree preflight.

Fetch and check out the week branch (or origin/main for a new week) before
running translate.py, and commit in place. The worklist now sees what is
already published, the untracked-file collision cannot happen, and content/en
is refreshed to the branch point in the same move.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
On the final round of the revise loop a page with open findings was revised
once more and then written — so the text published was post-revision while
the findings stamped on it, returned to the caller, and posted to the weekly
PR described the pre-revision text. The last Opus call per failing page
produced output nothing ever checked.

Stop revising once the rounds are exhausted: publish the text the reviewers
last saw, stamped -with-findings. The findings report now always corresponds
to the pages actually shipped, and no unreviewed revision can reach the tree.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
- Pin Python dependencies (requirements.txt): the runner pip-installs into
  its venv on every invocation on a machine holding maintainer credentials;
  unpinned installs would upgrade code in place next to those credentials.
- Drop || true on 'gh pr create' and 'gh pr comment': commits are already
  pushed at that point, so a failed gh call must fail the run — cron exit 0
  with pushed commits and no PR is indistinguishable from a good run.
- Remove the auto_merge code path entirely. 'Nothing reaches the production
  site without a maintainer merge' is meant as a property of the pipeline,
  not a default one config word away from off. Nothing used it.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
The disclaimer banner assembled one sentence from four i18n fragments plus
link text, hardcoding English word order and punctuation into the layout.
Two shipped languages already read wrong: Hindi is verb-final (the natural
order is 'अंग्रेज़ी मूल देखें', not 'देखें अंग्रेज़ी मूल'), and German/Russian continued
in lowercase after a question mark. German also split the article from its
noun phrase across two keys.

Use one key per sentence with the link as a Go-template placeholder, so each
translator places the link and punctuation where the language needs them.
Chinese now uses full-width punctuation without a space before the link.
These UI strings never pass through the pipeline's review gate (only
content/ does) — they are reviewed by hand.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
Deleting an English page left its translations behind forever: the worklist
only iterates English sources, so nothing ever revisited the leftovers, and
check-i18n.sh could only report them. translate.py now removes translations
whose English source is gone — only source_digest-stamped (pipeline-managed)
files; hand-authored locale-only pages are never touched — so the weekly PR
carries the deletion.

blog_since also accepts a rolling '<N>d' window (today minus N days,
resolved per run). The fixed ISO date would quietly rot into 'no blog post
is ever in scope' as the site ages; the config now uses '60d'. Aged-out
posts keep their existing translations and just stop being refreshed.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
Timur Tukaev (tym83) and others added 3 commits July 20, 2026 17:42
The README claimed a page is written 'only after it clears the gate', which
contradicted both the code and the rest of the document: pages that exhaust
the revise rounds are written and stamped -with-findings. Say plainly that
the gate triages rather than blocks, add the missing diagram edge for that
path, and note in Governance that it is a triage classifier, not a
gatekeeper.

Also: backlog math said 183 x 4 = 720 jobs while six languages are
configured (~1100 jobs, 55-220 h at the measured per-page rates); document
the rolling blog window and orphan removal; drop the duplicate .venv-i18n
gitignore entry and the last publish_mode reference.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
Three operational holes found in review of the runner reordering:

- A new week's branch was always cut from origin/main. If last week's PR is
  not merged yet (maintainers do skip weeks), its translations are not on
  main: the worklist would re-translate all of it on a day's quota and open
  a second PR conflicting with the first. Base a new week on the newest
  still-existing week branch instead; once the older PR merges, the new PR
  shrinks to its own commits.
- 'checkout -B origin/...' silently discarded local commits that a failed
  push left behind (GitHub outage on day 1 cost day 1's work). Keep a local
  week branch that is ahead of — or unknown to — origin.
- Orphan removal trusted the English tree unconditionally: a missing or
  renamed content/en would mark every stamped translation orphaned and
  commit a massacre. Refuse to run against a missing/empty English root,
  and refuse to remove more than 10 orphans at once — a large batch means
  the tree moved, not that ten pages died.

Also spell out crash recovery in the dirty-tree preflight message.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
Follow-ups from the second review round, all fail-safe refinements:

- Re-push any week branch that a failed push left stranded locally, at the
  start of every run. Waiting for new work to trigger the next push was not
  enough: with an empty backlog the run exits before publishing, and at a
  week boundary the new branch bases on origin refs — either way a
  locally-committed day sat unpublished and, across weeks, was re-translated.
  A branch missing on origin is only pushed when its PR state says it was
  never merged or closed: re-pushing a merged-and-deleted branch would
  resurrect it every morning and the new-week base selection would stack all
  later weeks on the zombie, freezing this clone's content/en at that week's
  snapshot. Merged/closed zombies are deleted locally instead.
- The mass-deletion floor now counts distinct English pages, not files: one
  deleted page fans out to one orphan per language, so a file-count floor
  tripped on two legitimately deleted pages across six languages.
- Tighten the week-branch glob to week-YYYY-WNN so a stray alphabetic branch
  (i18n/week-test) cannot sort above the dated ones and become a base.
- README: to discard a week, close the PR and delete the branch — the runner
  resumes on branch existence, not PR state.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
@netlify

netlify Bot commented Jul 20, 2026

Copy link
Copy Markdown

Deploy Preview for cozystack ready!

Name Link
🔨 Latest commit 12e90d5
🔍 Latest deploy log https://app.netlify.com/projects/cozystack/deploys/6a60b797ddbbbf0008f6a1a4
😎 Deploy Preview https://deploy-preview-623--cozystack.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@tym83

Copy link
Copy Markdown
Contributor Author

All findings addressed in 4653ba7..ae79b45 (8 commits). Point by point:

Runner wedge (blocking). Fixed the way you suggested: the week branch is now checked out before translating (fetch + checkout -B from origin/i18n/week-N or fallback base), and the run commits in place — 1f7f304. Reproduced your two-day scenario in a scratch repo before and after: day 2 now sees day 1's pages as done and the untracked-file collision cannot occur. Going further down that path surfaced three more holes in the reordering, fixed in 025d2a1 and ae79b45:

  • a new week always based on origin/main would re-translate an unmerged previous week and open a conflicting PR — a new week now bases on the newest surviving i18n/week-* branch;
  • checkout -B origin/... silently discarded commits stranded by a failed push — a local branch ahead of (or unknown to) origin is kept, and stranded branches are re-pushed at the start of every run;
  • the re-push logic in turn had to discriminate zombies: a merged-and-deleted branch whose local copy survived is deleted (by PR state via gh pr view, since ancestor checks break under squash merges), not resurrected. One residual edge remains: a morning where gh is broken but git push works could re-plant a zombie once; cheap hardening later is to skip that leg when gh auth status fails.

Revise loop. The final round no longer revises — 5c1269d. The text written is always the text the reviewers last saw, and the findings posted to the weekly PR describe the pages actually shipped.

README vs code. Reworded to say the gate triages rather than blocks, added the missing -with-findings edge to the diagram, fixed the job math (183 × 6 ≈ 1100), and noted in Governance that it's a triage classifier — 62395fc.

auto_merge is gone as a code path, not a default — b522046. || true on gh pr create/comment dropped in the same commit: commits are pushed by then, so a failed publish now fails the run. Deps pinned in hack/i18n/requirements.txt.

Banner. One i18n key per sentence with the link as a template placeholder, all seven languages — b90a72f. Hindi is verb-final now («अंग्रेज़ी मूल देखें»), de/ru capitalize after the question mark, zh-cn uses full-width punctuation. Verified with a full Hugo build rendering the banner in ru/de/hi/zh-cn. Agreed these strings bypass the content gate — they're hand-reviewed.

Smaller things45326b6, 62395fc: .gitignore dup removed; orphaned translations are now removed by the pipeline (only source_digest-stamped files, with a mass-deletion floor of 5 English pages and a hard refusal to run against a missing content/en); blog_since accepts a rolling "60d" window and the config uses it. One note: check-i18n.sh did already report orphans (references English source that does not exist) — what was missing was removal.

Tests: 54 pass (5 new), check-i18n.sh clean. Unrelated but spotted along the way: head-end.html drops the latest_version_id fallback — if that param ever disappears, all docs go noindex silently; happy to fix here or separately.

@IvanHunters

Copy link
Copy Markdown
Contributor

Verdict

LGTM with non-blocking notes

The core safety claims (code/shortcode byte-for-byte preservation, fail-closed reviewer-verdict parsing, no direct push to main, no auto-merge, es/pt-br declaration ordering) were independently re-derived from the code and executed, not just read, and all held up. Remaining items are non-blocking process/observability gaps.

Findings

[MINOR] hack/i18n/run-daily.sh:85-87,76-84 — no lock against overlapping invocations

The script's only defence against concurrent runs is a documented convention and a dirty-working-tree check. Two invocations against the same clone (a manual run overlapping a slow cron run) can both pass the clean-tree check, then race on git checkout -B "$BRANCH", commits, and git push. Worst case is a confusing local git state (no production impact, since nothing reaches main without an explicit PR merge), but it is silent until the second run's git push fails. A flock/pidfile guard at the top of the script would make this fail loudly and immediately instead of racing.

[MINOR] hack/i18n/translate.py (usage-limit stop / per-page skip-after-retry) — reported to stdout/stderr only, no durable record

RateLimited (daily quota hit) and repeated ProtocolError (page skipped after PROTOCOL_ATTEMPTS) are reported only via print(...). There is no GitHub Action (by design) and no log file recommended in the README's cron/launchd setup, so if the invoking job doesn't itself capture and monitor output, a page that fails every day, or a run that silently stops early on quota, has no durable signal for a maintainer. Not destructive (the page stays in the worklist and retries), but worth a one-line README addition (redirect to a rotated log, or a periodic worklist.py summary check).

Caveats

  • Hugo mount-for-undeclared-language mechanism verified in an isolated minimal Hugo build: a module.mounts entry for a language absent from languages: produces no output directory, so es/pt-br cannot publish empty indexable pages. A full hugo --gc --minify of the real tree could not complete in this environment (stalled on Hugo Modules resolution, unrelated to this diff); the targeted repro is what the verdict rests on.
  • bash hack/check-i18n.sh clean (key parity + freshness). python3 hack/i18n/test_i18n.py: 54/54 pass offline, including test_dropped_placeholder_is_rejected, test_duplicated_placeholder_is_rejected, test_unparseable_fails_closed, test_every_served_language_has_content.
  • "never publishes without a maintainer merge" traced end to end: every git push targets refs/heads/i18n/week-<ISO week>, gh pr create opens a PR, no gh pr merge/--auto anywhere. .github/CODEOWNERS unchanged in the final diff.
  • No committed/logged secrets: no workflow ships in this diff; CLAUDE_CODE_OAUTH_TOKEN/ANTHROPIC_API_KEY are read from environment only and never printed. .gitignore correctly ignores .venv-i18n/ and last-run-findings.md.
  • The two included sample pages contain no fenced code blocks, so they don't empirically exercise the "code preserved byte-for-byte" claim on real content — that guarantee rests on lib.py's protect/restore plus the placeholder-count check in translate.py, both read and both covered by passing unit tests. Samples do demonstrate inline-code and {{% ref %}}/{{% alert %}} shortcode preservation.
  • Prompt-injection surface: translate.py embeds raw English source into reviewer prompts unsanitized; mitigated by CODEOWNERS gating content/en/** behind human review before the pipeline sees it.

Recommended follow-ups

  • Add a flock/pidfile guard to run-daily.sh so a second concurrent invocation fails fast instead of racing on the same clone.
  • Document (README "Cadence") how cron/launchd output should be captured so a stuck run or a page repeatedly failing ProtocolError isn't missed.
  • Re-request review / dismiss the stale CHANGES_REQUESTED — the blocking runner bug and all secondary points from the prior review are addressed in the current HEAD.

Two non-blocking findings from review:

- A manual run overlapping a slow cron run passed the clean-tree preflight
  and then raced it on checkout/commit/push, silently until the second push
  failed. The script now takes a per-clone pidfile lock in TMPDIR and the
  second invocation fails fast and loudly. Pidfile rather than flock(1): the
  runner may be a macOS/launchd machine where flock does not exist. A lock
  whose pid is dead (crash, power loss) is taken over, not wedged on.
- The daily-quota stop and pages skipped on repeated protocol errors report
  only to stdout/stderr by design (no GitHub Action). README now shows the
  cron line with an append-only log and the two weekly health signals to
  glance at, so a page failing every day cannot stay invisible.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
@tym83

Copy link
Copy Markdown
Contributor Author

IvanHunters thanks for the deep pass — re-deriving the safety claims by execution is exactly the review this pipeline needed. Both follow-ups landed in 25732ae:

Overlap guard. Went with a pidfile rather than flock(1): the runner may well be a macOS/launchd machine, where flock doesn't exist. Per-clone lock in $TMPDIR (keyed by clone path, so two different clones on one box don't block each other), second invocation fails fast with the holder's pid; a lock whose pid is dead is taken over instead of wedging every subsequent cron run. Smoke-tested both branches: live pid blocks, dead pid takes over. Lock cleanup is folded into the existing EXIT trap (bash keeps only the last trap, so a separate one would have silently dropped the branch restore).

Log capture. README "Cadence" now shows the cron line with an append-only log and the two weekly health signals: grep -c '::warning::skipped' (a page stuck on protocol errors) and worklist.py | head (backlog must trend to zero). Agreed there's no durable channel by design — the log file is the durable channel now, and the README says so explicitly.

54/54 tests still pass, bash -n clean.

On the prompt-injection note: agreed the mitigation is CODEOWNERS on content/en/** — worth keeping in mind if the pipeline ever grows a path that translates content a human didn't merge (e.g. community PRs pre-merge previews). It doesn't have one today.

Andrei Kvapil (@kvaps) the blocking runner bug and all secondary points from your review are addressed in the current HEAD (4653ba7..25732ae, summary in my comment above) — could you re-review / lift the CHANGES_REQUESTED when you get a chance?

…ranslations build

A translated page cross-linking a sibling not yet translated into the same
language carried the English {{% ref %}} shortcode verbatim; Hugo resolves ref
within the current language and hard-fails REF_NOT_FOUND, breaking the whole
build (the failing Netlify deploy-preview). Add lib.deref_shortcodes (positional
+ named-param + fragment + anchor-only + relative + .md) plus has_ref_shortcode,
and fix the two shipped sample pages to plain absolute links, which render-link
resolves in-language or falls back to English (the 5ca8e55 convention).

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
…estable orphan floor

Apply deref to every written page and raise if any ref/relref survives, so an
unsupported shape is never shipped. Clear a stale last-run-findings.md on an
empty worklist (main returned before cleanup, so run-daily reposted resolved
findings). Surface early-stop/skipped pages in the durable report. Promote
ORPHAN_PAGE_FLOOR to a module constant and extract _distinct_orphan_pages so the
mass-deletion gating is unit-tested. Add tests for all of the above.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Persist each run's report to a state dir outside the repo tree (default
~/.local/state/cozystack-i18n) and post it on the nothing-to-publish path too, so
a run that stalled on the first day of a week is not invisible. Add publish_mode:
pr_only and fail closed in run-daily.sh on any other value. Correct the README
orphan mass-deletion floor (5 distinct pages, matching the code).

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
test_i18n.py (the regression guard for the ref build-break and other invariants)
ran nowhere in CI, and i18n-lint did not trigger on hack/i18n/. Add hack/i18n/**
to the path filters and a step that runs the tests (only pyyaml needed).

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
An unset latest_version_id made the not-latest comparison true for every version,
silently noindex-ing the entire docs tree including the current release. Fail
loudly at build time instead of relying on a silent, stale-prone default.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
…th the JSON code path

Harden _ref_target: strip backtick raw-string quotes and return None for any
target that still carries a quote/backtick/=/space, so an unparseable shape trips
the fail-closed guard instead of shipping a silently-broken link; map a section
index target (/_index, /index) to the parent URL Hugo actually serves. Rewrite
the translate/revise prompt Output protocol to describe the FRONTMATTER as the
JSON object the code sends and parses (the prose previously said key: value
lines, contradicting translate.py and risking wasted retries).

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOT LGTM — two write paths corrupt or silently downgrade pages, and both are reachable on the next run.

Business context: the English docs change constantly and native localization takes months, so this turns translation into a daily pipeline with a machine review gate and a weekly maintainer-merged PR, letting localized docs ship immediately with honest disclosure.

Ran the pipeline tests (74 pass), hack/check-i18n.sh (clean), and a production Hugo build to check the SEO changes. The runner rework holds up: the week branch is checked out before translating, the stranded-push and zombie-branch paths are handled, gh failures are no longer swallowed, and dependencies are pinned. My findings are in the translation core rather than the runner.

Blockers

B1: .html pages reach the model with their markup unprotected and unchecked

File: hack/i18n/lib.py:347

protect() masks fenced code, shortcodes, HTML comments and inline code. Raw HTML tags are not on that list, and .html pages are mostly raw HTML.

Evidence: running lib.protect() over the in-scope sources, content/en/docs/v1.5/roadmap.html yields zero protected spans — all three <a href="..."> links, the target="_blank", the mailto: and the <br /> go to the model verbatim. content/en/_index.html gets 25 spans (the shortcodes) but every <div class="hero-gitops">, </div> and <h2 class="section-label"> stays exposed. _split_payload_response (translate.py:217-223) only counts §§...§§ tokens, so with no tokens present there is nothing to fail closed on. worklist.py --lang ru lists docs/v1.5/roadmap.html today, so this is the next run and not a hypothetical.

Impact: a translated class= attribute, a dropped </div>, or a mangled URL ships silently. The gate does not check markup, check-i18n.sh only compares digests, and Hugo renders broken HTML without complaint. README.md:179 says URLs are preserved structurally, which holds for markdown links but not for these.

Fix: add an HTML-tag span to protect(), or drop .html from translate_globs and translate only their front matter.

B2: hand-written transcreations are overwritten with literal machine output

File: hack/i18n/translate.py:350

Evidence: eight pages carry l10n: transcreate from the proof-of-concept — content/{de,ru,hi,zh-cn}/_index.html plus four blog posts. _index.html is explicitly in translate_globs (config.yaml:107), and neither iter_source_files nor build_worklist consults l10n, so the page is re-translated the moment its English source changes. I checked that ru/_index.html's digest matches today, so staleness is one edit away. translate_page then replaces the body wholesale and writes out_fm["l10n"] = "mt" unconditionally, after the merge_target_only_keys call on line 344 — the marker the README lists as the human triage signal is the thing that gets clobbered.

Impact: four hand-crafted homepages become literal machine translation. Because the banner is wired into docs/baseof.html only, the localized homepage then carries that machine output with no disclosure at all. B1 applies to the same pages, so the markup is at risk in the same write.

Fix: skip a target page whose existing front matter says l10n: transcreate (or list those paths in exclude_globs) and report them in the weekly PR instead of overwriting them.

Non-blocking follow-ups

  1. run-daily.sh:212-214PREV_WEEK picks the newest remote week branch with no PR-state check. Merged branches are safe (delete_branch_on_merge is on and fetch --prune drops them), but closing a PR never deletes its branch, so rejecting a week's translations means next week bases on them and re-proposes the lot. The local-zombie path just above already asks gh pr view for state.
  2. run-daily.sh never checks gh up front. The first call is gh pr view ... || echo NONE (line 174), so an unauthenticated runner burns a full day of quota and pushes a branch before failing at gh pr create (line 292). A gh auth status alongside the claude/API-key preflight would fail in the first second instead.
  3. layouts/partials/hooks/head-end.html:50 uses .Language.LanguageCode, deprecated in Hugo v0.158.0 in favour of .Language.Locale; the pinned 0.160.1 warns on every build. Related: the two new module.mounts entries use lang:, deprecated in v0.153.0 in favour of sites.matrix. The lang: pattern is pre-existing, but these are new mounts.
  4. hack/i18n/config.yaml declares hreflang: per language while the template reads .Language.LanguageCode from hugo.yaml. Nothing consumes the config field, so either wire it up or drop it before it drifts.
  5. worklist.py has no --path, though its docstring claims flag parity and run-daily.sh forwards the same "$@" to both. The argparse error is swallowed by 2>/dev/null, so --path silently prints no preview.
  6. has_ref_shortcode runs after the whole gate, but it is a pure function of the English source since ref shortcodes are masked byte-for-byte. An unsupported shape therefore costs three full gates per language per day before anyone notices. All 315 refs across the 182 in-scope pages deref cleanly today, so this is a cost question rather than a correctness one.
  7. Once content/es/ lands without es in hugo.yaml, Hugo silently drops the mount — I built with a probe page and got no /es/ output and an untouched English tree. That is safe, but nothing signals that translated pages are accumulating unserved, and test_every_served_language_has_content only guards the other direction.
  8. The description still says this targets poc/i18n-multilang pending #593. That merged on 2026-07-20 and the base is already main.

Checked and dismissed: .Sites.Default is correct (it is the defaultContentLanguage site, available since Hugo 0.123, and resolves fine in the build); the duplicate .venv-i18n/ entry is gone; both prompts now specify the JSON front-matter protocol the code parses; and hreflang emits relative URLs in a bare local build but absolute ones under the production --baseURL, which is what Google needs.

Comment thread hack/i18n/lib.py
# ---- protect / restore ------------------------------------------------------


def protect(text: str) -> tuple[str, dict[str, str]]:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

protect() masks fences, shortcodes, HTML comments and inline code, but not raw HTML tags — and .html pages are mostly raw HTML.

Running this over the in-scope tree, content/en/docs/v1.5/roadmap.html produces zero protected spans: its three <a href="..."> links, the target="_blank", the mailto: and the <br /> all reach the model verbatim. content/en/_index.html gets 25 spans for its shortcodes, but every <div class="hero-gitops">, </div> and <h2 class="section-label"> stays exposed.

The placeholder guard in _split_payload_response only counts §§...§§ tokens, so on a page with no tokens there is nothing to fail closed on. A translated class= attribute, a dropped </div> or a mangled URL ships silently — the gate does not check markup, check-i18n.sh only compares digests, and Hugo renders broken HTML without complaint.

docs/v1.5/roadmap.html is in the worklist right now, so this lands on the next run.

Either add an HTML-tag span here, or drop .html from translate_globs and translate only their front matter.

Comment thread hack/i18n/translate.py
# (mt | transcreate). Whatever the page was before, this pipeline just
# machine-translated it, so say so — the disclaimer banner and any future
# native-review triage read this.
out_fm["l10n"] = "mt"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This overwrites l10n unconditionally, after the merge_target_only_keys call on line 344 — so the marker the README lists as the human triage signal is exactly what gets clobbered.

Eight pages carry l10n: transcreate from the proof-of-concept: content/{de,ru,hi,zh-cn}/_index.html plus four blog posts. _index.html is explicitly in translate_globs (config.yaml:107), and neither iter_source_files nor build_worklist consults l10n, so the page is re-translated as soon as its English source changes. ru/_index.html's digest matches today, which puts staleness one homepage edit away.

At that point four hand-crafted homepages become literal machine translation — and since the banner is wired into docs/baseof.html only, the localized homepage carries that output with no disclosure at all.

Suggest skipping a target page whose existing front matter says l10n: transcreate (or listing those paths in exclude_globs) and reporting them in the weekly PR instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants